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Abstract 



We investigate an optimal scheduling problem in a discrete-time system of L 
parallel queues that are served by K identical, randomly connected servers. 
Each queue may be connected to a subset of the K servers during any given 
time slot. This model has been widely used in studies of emerging 3G/4G 
wireless systems. We introduce the class of Most Balancing (MB) policies 
and provide their mathematical characterization. We prove that MB poli- 
cies are optimal; we define optimality as minimization, in stochastic ordering 
sense, of a range of cost functions of the queue lengths, including the process 
of total number of packets in the system. We use stochastic coupling argu- 
ments for our proof. We introduce the Least Connected Server First/Longest 
Connected Queue (LCSF/LCQ) policy as an easy-to-implement approxima- 
tion of MB policies. We conduct a simulation study to compare the perfor- 
mance of several policies. The simulation results show that: (a) in all cases, 
LCSF/LCQ approximations to the MB policies outperform the other poli- 
cies, (b) randomized policies perform fairly close to the optimal one, and, (c) 
the performance advantage of the optimal policy over the other simulated 
policies increases as the channel connectivity probability decreases and as 
the number of servers in the system increases. 



1 Introduction, Model Description and Prior 
Research 



Emerging 3G/4G wireless networks can be categorized as high-speed, IP- 
based, packet access networks. They utihze the channel variability, using 
data rate adaptation, and user diversity to increase their channel capacity. 
These systems usually employ a mixture of Time and Code Division Multiple 
Access (TDMA/CDMA) schemes. Time is divided into equal size slots, each 
of which can be allocated to one or more users. To optimize the use of the 
enhanced data rate, these systems allow several users to share the wireless 
channel simultaneously using CDMA. This will minimize the wasted capacity 
resulting from the allocation of the whole channel capacity to one user at a 
time even when that user is unable to utilize all of that capacity. Another 
reason for sharing system capacity between several users, at the same time 
slot, is that some of the user equipments at the receiving side might have 
design limitations on the amount of data they can receive and process at a 
given time. 

The connectivity of users to the base station in any wireless system is 
varying with time and can be best modeled as a random process. The appli- 
cation of stochastic modeling and queuing theory to model wireless systems 
is well vetted in the literature. Modeling wireless systems using parallel 
queues with random queue/server connectivity was used by Tassiulas and 
Ephremides [3] , Ganti, Modiano and Tsitsiklis |6] and many others to study 
scheduler optimization in wireless systems. In the following subsection, we 
provide a more formal model description and motivation for the problem at 
hand. 

1.1 Model Description 

In this work, we assume that time is slotted into equal length deterministic 
intervals. We model the wireless system under investigation as a set of L 
parallel queues with infinite capacity (see Figure [T]); the queues correspond 
to the different users in the system. We define Xi{n) to represent the number 
of packets in the i^^ queue at the beginning of time slot n. The queues share 
a set of K identical servers, each server representing a network resource, 
e.g., transmission channel. We make no assumption regarding the number of 
servers relative to the number of queues, i.e., K can be less, equal or greater 
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than L. The packets in this system are assumed to have constant length, 
and require one time slot to complete service. A server can serve one packet 
only at any given time slot. A server can only serve connected, non-empty 
queues. Therefore, the system can serve up to K packets during each time 
slot. Those packets may belong to one or several queues. 

The channel connectivity between a queue and any server is random. The 
state of the channel connecting the i*'^ queue to the j*^ server during the n*'* 
time slot is denoted by Gij{n) and can be either connected {Gij(n) = 1) 
or not connected {Gij{n) = 0). Therefore, in a real system Gij{n) will 
determine if transmission channel j can be used by user i or not. We assume 
that Gij{n), for alH = 1, 2, . . . , L, j = 1,2, ... ,K and n, are independent, 
Bernoulli random variables with parameter p. 

The number of arrivals to the i*'* queue during time slot n is denoted 
by Zi{n). The random variables Zi{n),^i,n is assumed to have Bernoulli 
distribution. We require that arrival processes to different queues be inde- 
pendent of each other; we further require that the random processes {Zj(n)} 
be independent of the processes {Gij{n)} ioi i = 1,2, . . . , L, j = 1,2, . . . , K. 
The symmetry and independence assumptions are necessary for the coupling 
arguments we use in our optimality proofs. The rest are simplifying assump- 
tions that can be relaxed at the price of a more complex and maybe less 
intuitive proof. 

A scheduling policy (or server allocation policy or scheduler) decides, at 
the beginning of each time slot, what servers will be assigned to which queue 
during that time slot. The objective of this work is to identify and analyze 
the optimal scheduling policy that minimizes, in a stochastic ordering sense, 
a range of cost functions of the system queue sizes, including the total number 
of queued packets, in the aforementioned system. The choice of the class of 
cost functions and the minimization process are discussed in detail in Section 

El 

1.2 Previous Work and Our Contributions 

In the literature, there is substantial research effort focusing on the subject 
of optimal scheduling in wireless networks with random connectivity. Tassi- 
ulas and Ephremides [3] studied the problem of allocating a single, randomly 
connected server to a set of parallel queues. They proved, using stochas- 
tic coupling arguments, that a LCQ (Longest Connected Queue) policy is 
optimal. In our work we investigate a more general model that studies the 
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Gi,i(n) Scheduler 




Figure 1: Abstraction of downlink scheduler in a multi-server wireless net- 
work. 

optimal allocation of K > 1 randomly connected servers to parallel queues. 
We show that LCQ is not always optimal in a multi-server system where mul- 
tiple servers can be allocated to each queue at any given time slot. Bambos 
and Michailidis [1] worked on a similar model (a continuous time version of [3] 
with finite buffer capacity) and proved that under stationary ergodic input 
job flow and modulation processes, both 'Maximum Connected Workload' 
and LCQ dynamic allocation policies maximize the stability region for this 
system. Furthermore, they proved that a policy that allocates the server to 
the connected queue with the fewest empty spaces, stochastically minimizes 
the loss flow and maximizes the throughput [5]. 

Another relevant result is that reported by Ganti, Modiano and Tsitsiklis 
[H]. They presented a model for a satellite node that has K transmitters. 
The system was modeled by a set of parallel queues with symmetrical statis- 
tics competing for K identical, randomly connected servers. At each time 
slot, no more than one server is allocated to each scheduled queue. They 
proved, using stochastic coupling arguments, that a policy that allocates the 
K servers to the K longest connected queues at each time slot, is optimal. 
This model is similar to the one we consider in this work, except that in our 
model one or more servers can be allocated to each queue in the system. A 
further, stronger difference between the two models is that we consider the 
case where each queue has independent connectivities to different servers. 
We make these assumptions for a more suitable representation of the 3G/4G 
wireless systems described earlier. These differences make it substantially 
harder to identify (and even describe) the optimal policy (see Section |3]). A 
more recent result that has relevance to our work is the one reported by Kit- 
tipiyakul and Javidi in |7|. They proved, using dynamic programming, that 
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a 'maximum-throughput and load-balancing' policy minimizes the expected 
average cost for a two-queue, multi-server system with random connectiv- 
ity. In our research work, we prove optimality of the most balancing policies 
in the more general problem of a multi-queue (more than two queues) and 
multi-server system with random channel connectivity. A stronger distinc- 
tion of our work is that we proved the optimality in a stochastic ordering 
sense which is a stronger notion of optimality compared to the expected av- 
erage cost criterion that was used in j7j . Lott and Teneketzis [8] investigated 
a multi-class system of weighted cost parallel queues and M servers with 
random connectivity. They also used the same restriction of one server per 
queue used in [H]. They showed that an index rule is optimal and provided 
conditions sufficient, but not necessary, to guarantee its optimality. 

Koole et al [9] studied a model similar to that of [3] and [5J. They found 
that the 'Best User' policy maximizes the expected discounted number of 
successful transmissions. Liu et al [10] , |TT] studied the optimality of oppor- 
tunistic schedulers (e.g.. Proportional Fair (PF) scheduler). They presented 
the characteristics and optimality conditions for such schedulers. However, 
Andrews [12] showed that there are six different implementation algorithms 
of a PF scheduler, none of which is stable. For more information on resource 
allocation and optimization in wireless networks the reader may consult [13] , 
[m, [S], [IS], [HI, and [IE]. 

The model we present in this work can be applied to many of the previous 
work described above. In section |9] we discuss this applicability for three key 
publications, namely [3], [6] and [7], that are strongly related to our own. 
We also show how our model can be reduced to their models and used to 
describe the problems they investigated. 

In summary, the main contributions of our work are the following: 

1. We introduce and show the existence of the class of Most Balancing 
(MB) scheduling policies in the model of Figure [l] (see Equations ([T]) 
and ([s])). Intuitively, an MB policy attempts to balance all queue sizes 
at every time slot, so that the total sum of queue size differences will 
be minimized. 

2. We prove the optimality of MB policies for minimizing, in stochastic 
ordering sense, a set of functionals of the queue lengths (see Theorem 

3. We provide low-overhead, heuristic approximations for an MB policy. 
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At any time slot, such policies allocate the "least connected servers 
first" to their "longest connected queues" (LCSF/LCQ). These policies 
have 0{L X K) complexity and thus can be easily implemented. We 
evaluate the performance of these approximations via simulations. 

The rest of the article is organized as follows. In section II, we introduce 
notation and define the scheduling policies. In section III, we introduce and 
provide a detailed description of the MB policies. In section IV, we intro- 
duce and characterize balancing interchanges, that we will use in the proof 
of MB optimality. In section V, we present the main result, i.e., the opti- 
mality of MB policies. In section VI, we present the Least Balancing (LB) 
policies, and show that these policies perform the worst among all work con- 
serving policies. MB and LB policies provide upper and lower performance 
bounds. In section VII, we introduce practical, low-overhead approximations 
for such policies, namely the LCSF/LCQ policy and the MCSF/SCQ policy, 
with their implementation algorithms. In section VIII, we present simula- 
tion results for different scheduling policies. In section IX, we give some 
final remarks that show the applicability of our model to problems studied 
in previous work. We present proofs for some of our results in the Appendix. 

2 Scheduling Policies 

Recall that L and K denote the number of queues and servers respectively 
in the model introduced in Figure [TJ We will use bold face, UPPER CASE 
and lower case letters to represent vector/matrix quantities, random variables 
and sample values respectively. In order to represent the policy action that 
corresponds to "idling" a server, we introduce a special, "dummy" queue 
which is denoted as queue 0. Allocating a server to this queue is equivalent 
to idling that server. By default, queue is permanently connected to all 
servers and contains only "dummy" packets. Let denote the indicator 
function for condition A. Throughout this article, we will use the following 
notation: 

• G{n) is an (L + 1) X matrix, where Gij{n) for i > is the channel 
connectivity random variable as defined in Section [Tj By assumption, 
Goj{n) = 1 for all j,n. 

• X(n) = (Xo(n), X2(n), Xi(n))-^ is the vector of queue lengths 



5 



at the beginning of time slot n, measured in number of packets. We 
assume Xo{l) = 0. 

• Y{n) = (10(^)5 ^i(^)? Y2{n), . . . , Yl^u))'^ is the withdrawal control. For 
any i, Yi{n) G {0, 1, ... , K} denotes the number of packets withdrawn 
from queue i (and assigned to servers) during time slot n. 

• Z(n) = {Zo{n), Zi{n), Z2{n), . . . , Zl^u))'^ is the vector of the number 
of exogenous arrivals during time slot n. Arrivals to queue i ^ are as 
defined in Section [U 

• For ease of reference, we call the tuple (X(?7,), G{n)) the "state" of the 
system at the beginning of time slot n. 

For any (feasible) control (Y(n)), the system described previously evolves 
according to 

X(n + 1) =X(n)-Y(n) + Z(n), n = l,2,... (1) 

We assume that arrivals during time slot n are added after removing 
served packets. Therefore, packets that arrive during time slot n have no 
effect on the controller decision at that time slot and may only be withdrawn 
during t = n + 1 or later. For convenience and in order to ensure that 
Xo(n) = for all n, we define Zq^u) = Yq^u). We define controller policies 
more formally next. 

2.1 Feasible Scheduling and Withdrawal Controls 

The withdrawal control defined earlier does not provide any information re- 
garding server allocation. Such information is necessary for our optimal- 
ity proof. To capture such information, we define the vector Q(n), where 
Qj{n) e {0,1,..., L} denotes the index of the queue that is selected (ac- 
cording to some rule) to be served by server j during time slot n. Note that 
serving the "dummy" queue, i.e., setting Qj{n) = indicates that server 
j is idling during time slot n. For future reference, we will call Q(?t.) the 
scheduling (or server allocation) control. 

Using the previous notation and given a scheduling control vector Q(n) 
we can compute the withdrawal control vector as: 

K 

Y,{n) = J2 l{.=Q.(n)}, I = 0,1,2,..., L. (2) 
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We say that a given vector Q{n) G {0, 1, . . . , L}^ is a feasible scheduling 
control (during time slot n) if: (a) a server is allocated to a connected queue, 
and, (b) the number of servers allocated to a queue (dummy queue excluded) 
cannot exceed the size of the queue at time n. Similarly, we say that a vector 
Y(n) e {0, 1, . . . , K}^~^^ is a feasible withdrawal control (during time slot n) 
if there exists a feasible scheduling control Q{n) that satisfies Equation (|2]). 

Conditions (a) and (b) above are also necessary for feasibility of a schedul- 
ing control vector Q(n). From Equation ([2]), a feasible withdrawal control 
Y(n) satisfies the following necessary conditions: 

<Yi{n) < min (^X,{n), GiA^^j , Vn, z 0, (3) 

L 

Y,yi{n)=K, Vn. (4) 

For the rest of this article, we will refer to Q(n) as an implementation of 
the given feasible control Y(n). We denote the set of all feasible withdrawal 
controls while in state (x, g) by 3^(x, g). 

Note from Equation ^ that, given a feasible scheduling control Q,{ti), a 
feasible withdrawal control Y{n) can be readily constructed. Note, however, 
that, for any feasible Y(n), the feasible scheduling control Q(n) may not be 
unique. Furthermore, given a feasible Y(n), the construction of the schedul- 
ing control Q{n) may not be straightforwarc^ and will not be examined in 
this article. 



2.2 Definition of Scheduling Policies 

A scheduling policy vr (or policy tt for simplicity) is a rule that determines 
feasible withdrawal vectors Y(n) for all function of the past history 

and current state of the system H(?t,). The state history is given by the 
sequence of random variables 

H(l) = (X(l)), and 

H(n) = (X(l),G(l),Z(l),...,G(n-l),Z(n-l),G(n)), 

n = 2,3,... (5) 

^ Given a state (X(n), G{n)) and a feasible withdrawal vector Y(?i), one can determine 
the feasible scheduling control by performing a brute-force search over all feasible vectors 
Q(n). 
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Let 1-in be the set of all state histories up to time slot n. Then a policy 
TT can be formally defined as the sequence of measurable functions 

„ ■ a/ I i. ^L+i 

s.t. u„(HH) G3^(XH,G(n)), n = l,2,... (6) 

where ^+ is the set of non-negative integers and Z^'^^ = x • • ■ x 
where the Cartesian product is taken L + 1 times. 

At each time slot, the following sequence of events happens: first, the 
connectivities G{n) and the queue lengths X(n) are observed. Second, the 
packet withdrawal vector Y(n) is determined according to a given policy. 
Finally, the new arrivals Z(ri) are added to determine the next queue length 
vector X(n + 1). 

We denote the set of all scheduling policies described by Equation ^ 
by n. We introduce next a subset of 11, namely the class of Most Balancing 
(MB) policies. The goal of this work is to prove that MB policies are optimal 
(in a stochastic ordering sense). 

3 The Class of MB Policies 

In this section, we provide a description and mathematical characterization 
of the class of MB policies. Intuitively, the MB policies "attempt to minimize 
the queue length differences in the system at every time slot n" . For a more 
formal characterization of MB policies, we first define the following: 

Given a state (x(n), g(n)) and a policy vr that chooses the feasible control 
y{n) G 3^(x, g) at time slot n, define the "updated queue size" Xiiji) = 
Xiin) — Uiin) as the size of queue i = 0, 1, . . . , L, after applying the control 
Uiin) and just before adding the arrivals during time slot n. Note that because 
we let zoiji) = yo{n), we have xo{n) G Z where Z is the set of all integers, 
i.e., we allow xo{n) to be negative. 

We define Kn(7r), the "imbalance index" of policy n at time slot n, as the 
following sum of differences: 

L L+l 
1=1 j=i+l 

where [k] denotes the index of the k^'^ longest queue after applying the con- 
trol y{n) and before adding the arrivals at time slot n. By convention, queue 
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'0' (the "dummy queue") will always have order L + 1 (i.e., the queue with 
the minimum length). This definition ensures that the differences are non- 
negative and a pair of queues is accounted for in the summation only once; 



moreover, as we shall see in Lemma [Appendix .2.1[ this definition allows for 



a straightforward calculation and comparison of various policie^ 

It follows from Equation ([T]) that the minimum possible value of the 
imbalance index is equal to L ■ X[l] (i.e., all L queues have the same length 
which is equal to the shortest queue length) which is indicative of a fully 
balanced system. It also follows that the maximum such value is equal to 
2(L — l)x[i] — (L — 2)f [i]. This value is attained when the L — 1 longest 
queues have the same size. 

Let Yl^^ denote the set of all MB policies, then we define the elements 
of this set as follows: 

Definition: A Most Balancing (MB) policy is a policy tt G II^-'^'^ that, at 
every n = 1,2, . . ., chooses feasible withdrawal vector y{n) G 3^(x, g) such 
that the imbalance index at that time slot is minimized, i.e., 

n*^^ = {tt G n : argmin fi:„(7r), Vn} (8) 

y(n)ey(x,g) 



The set n*''^'^ in Equation ([s]) is well-defined and non-empty, since the 
minimization is over a finite set. Note that the set of MB policies may have 
more than one element. This could happen, for example, when at a given 
time slot n, a server k is connected to two or more queues of equal size, which 
happen to be the longest queues connected to this server after allocating all 
the other servers. To illustrate this case, consider a two-queue system with 
a single, fully-connected server at time slot n. Let x(n) = (5,5). Assume 
that policy tti (respectively 112) chooses a withdrawal vector y(n) = (1,0) 
(respectively y*{'n) = (0,1)). Then both policies minimize the imbalance 
index, and Uni^^i) = '^n{''^2) = 10. 

Given X(t) and G(t), one can construct an MB policy using a direct 
search over all possible server allocations. For large L and K, this can be a 
challenging computational task and is not the focus of this work. In Section 
[7| we provide a low-complexity heuristic algorithm (LCSF/LCQ) to approx- 
imate MB policies. 



^We have experimented with alternatives to Equation ^ that use lexicographic order- 
ing of queues and "Min-Max" definitions (e.g., minimize the length of the largest queue). 



However, we were not able to derive results equivalent to Lemma Appendix .2.1 
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Remark 1. Note that the LCQ policy in is a most balancing (MB) policy 
for K = 1 (i.e., the one server system presented in fB^). Extension of LCQ to 
K > 1 (i.e., allocating all the servers to the longest queue in the multiserver 
model) may not result in a MB policy, as the following example demonstrates. 

Consider a system of three queues with three fully- connected servers dur- 
ing time slot n. Let x(n) = (6, 5, 4). An LCQ policy in the spirit of that 
allocates all servers to the longest connected queue results in queue size vector 
x(n) = (3,5,4). Moreover, an LCQ policy in the spirit of IB] that allocates 
the three servers to the three longest connected queues results in queue size 
vector x(n) = (5, 4, 3). Both policies have i^nij^) = 16. An MB policy results 
in queue size vector 'k{n) = (4,4,4) and Uni'^) = 16. □ 



3.1 Comparing arbitrary policies to an MB policy 

When comparing various policies to an MB policy, the definition in Equation 
^ is cumbersome since it involves all time instants n. The subsets n„ we 
introduce next define policies that are related to MB policies and allow us to 
perform comparisons one single instant at a time. 

Consider any fixed n > 1; we say that a policy vr G 11 "has the MB 
property" at time n, if vr achieves the minimum value of the index K„(7r). 

Definition: For any given time n > 0, n„ denotes the set of policies that 
have the MB property at all time slots t < n (and are arbitrary for t > n). 

We have that 11 = IIo. Note that the set n„ is not empty, since MB 
policies are elements of it. We can easily see that these sets form a monotone 
sequence, with 

n„ c n„_i (9) 

Then the set in Equation p| can be defined as 



n=l 



The vector D defined in Equation (10) is a measure of how much an 
arbitrary policy vr differs from a given MB policy during a given time slot n. 

Definition: Consider a given state (x(n), g(n)) and a policy vr that 
chooses the feasible withdrawal vector y{n) during time slot n. Let y'^^^{n) 
be a withdrawal vector chosen by an MB policy during the same time slot n. 
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We define the (L + 1) x 1-dimensional vector D G Z^^^ as 

D = y^^^(n)-y(n). (10) 

Note that, for notational simphcity, we omit the dependence of D on the 
policies and the time index n. Intuitively, a negative element Di of vector D 
indicates that more packets than necessary (compared to a policy that has 
the MB property) have been removed from queue i under policy vr. 

The following lemma quantifies the difference between an arbitrary policy 



and an MB policy (at time n). Its proof is given in Appendix Appendix .1 



Lemma 1. Consider a given state (x(r;,), g(n)) and a policy vr G 11. Then, 
(a) ifT) = 0, the policy tt has the MB property at time n, and, (h) if tt has 
the MB property at time n, the vector D has components that are 0, +1, or 
— 1 only. 

Consider a policy tt G 11; let h^^ = Yli=o As we show in the 



Appendix (see Lemma Appendix .3.2), /i^r is integer-valued and < h^^ < K. 



In view of Lemma [TJ h.^ can be seen as a measure of "how close" the policy 
TT is to having the MB property at time n. 

Definition: For any given time n and integer h, where < h < K, define 
the set as the set that contains all policies vr G n„_i, such that /i^ < h. 

From Lemma [T| we can see that 11° = n„. We can easily check that 
= n„_i, so TT G by default. {H^j^Q forms a monotone sequence, 

with 



n„ = no c...cn::c...cn^ = n„_i (ii) 

We exploit the monotonicity property of the 11^ sets in the next section, 
when we show how balancing interchanges reduce the imbalance index of a 
given policy. 

Note that the set 11 of all policies can be denoted as 



n = Ho = Uto^l (12) 

It follows from the last two equations that an arbitrary policy vr G 11 
will also belong to a set n„_i, for some n > 1. The proof of optimality in 
Section [5] is based on comparisons of tt to a series of policies that belong to 
the subsets 11^ (see Lemma p|. 
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4 Balancing Interchanges 



In this section, we introduce the notion of "balancing interchanges". In- 
tuitively, an interchange l{f,t) between two queues, / and t, describes the 
action of withdrawing a packet from queue / instead of queue t (see Equations 
(15) and (16)). Such interchanges are used to relate the imbalance indices 
of various policies (see Equation (23)); balancing interchanges are special in 
two ways: (a) they do not increase the imbalance index (see Lemma [2]) and 
thus provide a means to describe how a policy can be modified to obtain the 
MB property at time n, and, (b) they preserve the queue size ordering we 
define in the next section (see relations R1-R3 in Section 5.1). This ordering 
is crucial in proving optimality. 

Interchanges can be implemented via server reallocation. Since there are 
K servers, it is intuitive that at most K interchanges suffice to convert any 
arbitrary policy to a policy that has the MB property at time n. The crux 
of Lemma |4| the main result of this section, is that such interchanges are 
balancing. 



4.1 Interchanges between two queues 

Let / G {0, 1, ... , L}, t G {0, 1, . . . , L} represent the indices of two queues 
that we refer to as the 'from' and 'to' queues. Define the (L + 1) x 1- 
dimensional vector I{f,t), whose j-th element is given by: 

0, t = f; 

+ 1, J = f, f T^t; , . 

-1, J=t,t^f; 
0, otherwise. 

Fix an initial state (x(n), g{n)) at time slot n; consider a policy vr with a 
(feasible) withdrawal vector y{n). Let 

y*{n)=yin)+I{f,t), f ^ t, (14) 

be another withdrawal vector. The two vectors y{n),y*{n) differ only in 
the two components t,f] under the withdrawal vector y*(n), an additional 
packet is removed from queue /, while one packet less is removed from queue 
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t. Note that either t or / can be the dummy queue. In other words, 



y}in) = y/H + l (15) 
y;{n) = yt{n)-l (16) 
y*{n) = y,{n), (17) 

In the sequel, we will call I(/, t) an interchange between queues / and t. 
We will call I(/, t) a feasible interchange if it results in a feasible withdrawal 
vector y*(n). It follows immediately from Equations ([T]) and (14) that the 



I(/, t) interchange will result in a new vector, x*(ra), of updated queue sizes, 
such that: 

x*(n)=x(n)-I(/,t), f^t. (18) 

We are interested next in describing sufficient conditions for ensuring 
feasible interchanges. 



4.2 Feasible Single-Server Reallocation 

Given the state (x(n),g(n)), let y(n) be any feasible withdrawal vector at 
time slot n that is implemented via q(n). We define a "feasible, single-server 
reallocation" (from queue t to queue /) as the reallocation of a single server 
k from queue t to queue /, such that the new scheduling control q*(n) is 
also feasible. The conditions Qf^kip) ■ gt,k{n) ■ = 1 and Xf{n) > 1 

are sufficient for the reallocation of server k (from queue t to queue /) to be 
feasible. 

A feasible, single-server reallocation from queue t to queue / results into 
a feasible interchange I{f,t). However, the reverse may not be true, as we 
detail in the following section. 



4.3 Sufficient conditions for a feasible interchange 

Consider again the state (x(n),g(n)) and feasible scheduling control q(n). 
The feasible interchange I(/, t) in Equation ( 14 ) may result from a sequence of 
m feasible, single-server reallocations among several queues, as demonstrated 
in Figure [2| where 1 < m < K. 

Let r G {0, 1, . . . , denote a sequence of queue indices, where ri = / 

and r^+i = t. Let ki : ki G {1, 2, ... , K} denote the server reallocated from 
queue rj+i to queue rj. Then the following are sufficient conditions for the 
feasibility of the interchange operation of Equation (14): 



13 



Figure 2: A sequence of m single-server reallocations results in a feasible 
I{f,t). The dotted line denotes original server allocation. The solid line 
denotes server reallocation that implements t). 



^^dnMi^) ■ 9r,+ukX'n) ■ l{gfcjn)=n+i} = m, (19) 



i=l 



Xfin)>l, if /G{1,2,...,L}, (20) 



for some integer 1 < m < K and r G {0, 1, . . . , L} 



m+l 



Constraint ( 19 ) ensures that connectivity conditions allow for the feasibil- 
ity of all m intermediate single-server reallocations. The sequence of server 
reallocations starts by reallocating server ki to queue f = ri. In this case, 
queue ri is reduced by one packet (i.e., an extra packet is withdrawn from 



queue /) and queue r2 is increased by one packet. Constraint (20) ensures 
that a packet can be withdrawn from queue /. The reallocation of server ki 
insures that queue r2 contains at least one packet for the second intermediate 
single-server reallocation to be feasible even when Xr2{n) = 0. Same is true 
for any queue rj : i G {2, 3, . . . , m}. Therefore, constraints ( |l9| and (20) are 
also sufficient for the feasibility of the interchange I{f,t). 

4.4 "Balancing" interchanges 

Definition: A feasible interchange I(/, t) is ^^balancing" if 

Xfin) > xtin) + 1. (21) 
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t) is ^^unbalancing" if 



Xf{n)<xtin). (22) 

Balancing interchanges result in policies that may reduce the imbalance 
index, as the following lemma states. 

Lemma 2. Consider two policies vr* and tt, related via the balancing inter- 
change 

y*H = y(n)+I(/,t), 
at time slot n. Then the imbalance indices for the two policies are related via 

«n(71"*) = Kn(7r) - 2(s - /) ■ {n)>£[,] (n)+2} (23) 

where I (respectively s) is the order of queue f (respectively t) in x{n) when 
ordered in descending order, such that, s > l;xi > Xa-,^a > I and Xs < 

Xb,yb < sj5 



The proof is a direct consequence of Lemma [Appendix .2.11 in Appendix 



Appendix .2 and the fact that, by definition of the balancing interchange, we 



have s > 1. 

In words. Equation (23) states that an interchange I{f,t), when balanc- 
ing, results in: either a cost reduction of 2(s — /) (when Xf{n) = X[i]{n) > 
3^[s](^) + 2 = Xt{n) + 2) or an unchanged cost (when Xf{n) = Xt{n) + 1). The 
latter case agrees with intuition, since the balancing interchange in this case 
will result in simply permuting the lengths of queues / and t; this permuta- 
tion does not change the total sum of differences (and hence the imbalance 
index) in the resulting queue length vector. 

We determine next conditions that characterize what interchanges are 
balancing. We also describe how balancing interchanges transform an arbi- 
trary policy to an MB policy. 

4.5 How to determine balancing interchanges 

Lemmajsjprovides a selection criterion to systematically select balancing (and 
hence improving) interchanges. Lemma |4] provides a bound on the number of 

•^These conditions state that when there exist muhiple components that have the same 
value as xi (respectively Xg) only the last (respectively the first) of the components in order 
is considered. Intuitively, we use s (respectively I) to refer to the order of the "shorter" 
(respectively the "longer") queue of the two queues used in the interchange. 
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interchanges needed to convert any policy into one that has the MB property 
at time n. The proofs of the two lemmas are given in Appendix Appendix 
^ and Appendix .41 respectively. 



Lemma 3. Consider a given state (x(n),g(n)) and a feasible withdrawal 
vector y{n). Any feasible interchange l{f,t) with indices f and t such that 
Df > +1, Dt < —1 is a balancing interchange. 

Recall that h.^ = Xll^o l-^^l/^- Consider a sequence of balancing inter- 
changes, I(/i,ti),I(/2,t2), • • . Let 

y*{n)=y{n) + J2m,U). 

i=l 

We denote by vr* the policy that chooses the withdrawal vector y*{n). In 
other words, vr* denotes the pohcy that results from applying this sequence 
of interchanges. 

Lemma 4. For any policy vr G Hn-i, /^tt balancing interchanges suffice to 
determine a policy tt* such that vr* G n„. 

Lemma [3] can be used to identify queues fi and during time slot n such 
that the interchange is balancing. Lemma |4] shows that performing a 

sequence of such interchanges, determines a policy that has the MB property 
for one more time slot. Both lemmas are crucial for the proof of our main 
result, since they indicate how a given policy can be improved using one 
balancing interchange at a time. 

5 Optimality of MB Policies 

In this section, we present the main result of this article, that is, the opti- 
mality of the Most Balancing (MB) policies. We will establish optimality 
for a range of performance criteria, including the minimization of the total 
number of packets in the system. We introduce the following definition. 

5.1 Definition of Preferred Order 

Let's define the relation ^ on Z^'^^'' first; we say x ^ x if: 
Rl- Xi < Xi for all i (i.e., point wise comparison). 
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R2- X is obtained from x by permuting two of its components; the two 
vectors differ only in two components i and j, such that = xj and 

cc j — Ob 2 f or 

R3- X is obtained from x by performing a balancing interchange", in the 



sense of Equation (21), i.e., the two vectors differ in two components 
i > and j > only, where Xi > a;^ + 1, such that: Xi = Xi — 1 and 
X j — X j I X . 

To prove the optimality of MB policies, we will need a methodology that 
enables comparison of the queue lengths under different policies. Towards 
this end, we define a "preferred order" as follows: 

Definition: (Preferred Order). The transitive closure of the relation ^ 
defines a partial order (which we call preferred order and use the symbol -<p 
to represent) on the set z'^^^\ □ 
The transitive closure El] , |6] of ^ on the set Z^^^^ is the smallest tran- 



sitive relation on Z^_^^^'^ that contains the relation From the engineering 
point of view, x -<p x if x is obtained from x by performing a sequence of 
reductions, permutations of two components and/or balancing interchanges. 

For example, if x = (3, 4, 5) and x = (4, 5, 3) then x -<p x since x can be 
obtained from x by performing the following two consecutive two-component 
permutations: first swap the second and third components of x, yielding 
x^ = (4,3,5) then swap the first and second components of x^, yielding 
x2 = (3,4,5) = i. 

Suppose that x, x represent queue size vectors for our model. Statement 
R3 in this case describes moving a packet from one real, large queue i to 
another smaller one j (note that the queue with index j = is not excluded 
since a balancing interchange may represent the allocation of an idled server). 
We say that x is more balanced than x when R3 is satisfied. For example, if 
L = 2 and x = (0, 5, 2) then a balancing interchange (where i = 1 and j = 2) 
will result in x = (0, 4, 3). 



5.2 The class T of cost functions 

Let X, X G Z^_^^^^ be two vectors representing queue lengths. Then we denote 
by J-' the class of real-valued functions on Z^^^^ that are monotone, non- 
decreasing with respect to the partial order -<p; that is, / G J-" if and only 
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if 

i^px ^/(5)</(x). (24) 

From ( [24) ) and the definition of preferred order, it can be easily seen that 
the function /(x) = Xi+X2 + - ■ ■+Xl belongs to J-". This function corresponds 
to the total number of queued packets in the system]^ 

For two real- valued random variables A and B, A <st B defines the usual 
stochastic ordering [2] . In the remainder of this paper, we say that a policy 
a dominates another policy vr if 

/(X-(t)) /(X-(t)), Vt = l,2,... (25) 

for all cost functions f ^ T . 

We will need the following lemma to complete the proof of our main result 
presented in Theorem [T] 

Lemma 5. Consider an arbitrary 'policy vr G 11^, inhere h > 0. Then, there 
exists a policy vf G n^~^, such that vf dominates vr. 



The full details of the proof for Lemma |5] are given in Appendix Appendix 



The proof involves two parts. First, we construct a policy tt by applying 
a balancing interchange to vr; using Lemmas and 0, we show that tt G H^j:'^. 



Second, we prove that tt dominates policy vr (see Equation (25)); this part 
employs coupling arguments. 



5.3 The main result 

In the following, X^^^ and X'^ represent the queue sizes under a MB and an 
arbitrary policy vr. 

Theorem 1. Consider a system of L queues served by K identical servers, 
as shown in Figure [I] with the assumptions of Section [1} A Most Balancing 
(MB) policy dominates any arbitrary policy when applied to this system, i.e., 

/(X^^^(t)) /(X-(t)), Vt = l,2,... (26) 

for a// vr G n and all cost functions f & T . 

''Another example is the function /'(x) = max{a;i, . . . which also belongs to the 
class F. 
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Proof. From (24) and the definition of stochastic dominance, it is sufficient to 
show that X^^^t) -<p X'^(t) for all t and all sample paths in a suitable sample 
space. The sample space is the standard one used in stochastic coupling 



methods [I]; see Appendix Appendix .5 for more details. 

To prove the optimality of an MB policy, vr*^^, we start with an arbitrary 
policy TT and apply a series of modifications that result in a sequence of 
policies (vTi, 7r2, . . .). The modified policies have the following properties: 

(a) TTi dominates the given policy vr, 

(b) TTj G Ilj, i.e., policy vTj has the MB property at time slots t = 1,2, ... ,i, 
and, 



(c) TTj dominates tTj for j > i (i.e., ttj has the MB property for a longer 
period of time than tTj). 



Let 71 be any arbitrary policy; then ir eHq = II f . Using Lemma [s] we can 
construct a policy vr G Llf""^ that dominates the original policy tt. Repeating 
this operation we can construct policies that belong to Llf'"^, nf^~^, ...,115* = 
Hi such that all dominate the original policy vr. This sequence of construction 
steps will result in a policy vri that has the MB property at t = 1, i.e., 
vTi G Hi, and dominates vr. Therefore, by construction vri G Llf-. We repeat 
the construction steps above for time slot t = 2, by improving on tti, to 
obtain a policy it2 G 112 that dominates tti, and recursively for t = 3, 4, . . . to 
obtain policies ir^, 114, .. . . From the construction of 7r„, n = 1, 2, . . ., we can 
see that it satisfies properties (a), (b) and (c) above. 

Denote the limiting policy as n — > 00 by vr*. One can see that n* is 
an MB policy. Furthermore, vr* dominates tTj, for all i < 00, as well as the 
original policy vr. □ 



Remark 2. The optimal policy may not be unique. Our main objective is 
to prove the optimality of the MB policy not its uniqueness. The optimality 
of MB policies makes intuitive sense; any such policy will tend to reduce the 
chance that any server idles. This is because an MB policy distributes the 
servers among the connected queues in the system such that it keeps packets 
spread among all the queues in a "uniform" manner. □ 
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6 The Least Balancing Policies 



The Least Balancing (LB) policies are the scheduhng pohcies, among all 
work- conserving (non-idling) policies, that at every time slot (n = 1,2, . . .), 
choose a packet withdrawal vector y{n) G 3^(x, g) that "maximizes the differ- 
ences" between queue lengths in the system (i.e., maximizes K„(7r) in Equa- 
tion ([7|). In other words, if 11^^ is the set of all LB policies and n^*^ is the 
set of all work conserving policies then, 

U^^ = {n: argmax K„(7r),7r G n^^, Vn} (27) 

y(n)6y(x,g) 

Maximizing the imbalance among the queues in the system will result in 
maximizing the number of empty queues at any time slot, thus maximizing 
the chance that servers are forced to idle in future time slots. This intuitively 
suggests that LB policies will be outperformed by any work conserving policy. 
The next theorem states this fact. Its proof is analogous to that of Theorem 
[T]and will not be given here. 

Remark 3. A non-work conserving policy can by constructed such that it 
will perform worse than LB policies, e.g., a policy that idles all servers. □ 

Theorem 2. Consider a system of L queues served by K identical servers, 
under the assumptions described in Sections [7| A Least Balancing (LB) 
policy is dominated by any arbitrary work conserving policy when applied to 
this system, i.e., 

/(X-(t)) /(X^^(t)), Vt = l,2,... (28) 
for all n G Yl^^ and all cost functions / G J-". 

An LB policy has no practical significance, since it maximizes the cost 
functions presented earlier. Intuitively, it should also worsen the system 
stability region and hence the system throughput. However, it is interesting 
to study the worst possible policy behavior and to measure its performance. 
The LB and MB policies provide lower and upper limits to the performance of 
any work conserving policy. The performance of any policy can be measured 
by the deviation of its behavior from that of the MB and LB policies. 
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7 Heuristic Implementation Algorithms For 
MB and LB Policies 



In this section, we present two heuristic pohcies that approximate the behav- 
ior of the MB and LB pohcies respectively. We present an implementation 
algorithm for each one of them. 

7.1 Approximate Implementation of MB Policies 

We introduce the Least Connected Server First/Longest Connected Queue 
(LCSF/LCQ) pohcy, a low-overhead approximation of MB policy, with 0{Lx 
K) computational complexity. The policy is stationary and depends only on 
the current state (X(n),G(n)) during time slot n. 

The LCSF/LCQ implementation during a given time slot is described as 
follows: The least connected server is identified and is allocated to its longest 
connected queue. The queue length is updated (i.e., decremented). Wc pro- 
ceed accordingly to the next least connected server until all servers are as- 
signed. In algorithmic terms, the LCSF/LCQ policy can be described/implemented 
as follows: 

Let Qj = {i : i = 1,2, ... ,L; Qijit) = 1} denote the set of queues that are 
connected to server j during time slot t; we omit the dependence on t to 
simphfy notation. Let Q[j] be the i*^ element in the sequence (Qi, . . . , Q,k): 
when ordered in ascending manner according to their size (set cardinality), 
i.e., |Q[;]| > |Q[m]| if / > m. Ties are broken arbitrarily. Then under the 
LCSF/LCQ policy, the K servers are allocated according to the following 
algorithm: 
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Algorithm 1 (LCSF/LCQ Implementation). 

1. for t = 1,2,... do I 

2. Input: X(t), G{t). Calculate Q[i], l = l,...,K. 

3. X' < — X(t), Y ^ — 0, Q ^ — 

4. for j = 1 to K I ; allocate servers sequentially 

5. Q[j] = min ( I : I e I argmax(X^|X^ > 0) M 

6. for i = 1 to L I 

7. i'. = i'. + 

8. A-,' = A-,(«) - } } 

9. Output: y(t) ^ — Y, q(t) < — Q ; report outputs 
10. I ; End of Algorithm^ 

Note that in line 5 of Algorithm [T| if the set Qyj is empty, then the argmax 
returns the empty set. In this case, the j^^ order server will not be allocated 
(i.e., will be idle during time slot t). Algorithm [l] produces two outputs, 
when it is run at t = n: y(n) and ci{n) as shown in line 9 of the algorithm. 
In accordance to the definition of a policy in Equation (|6]), the LCSF/LCQ 
policy can be formally defined as the sequence of time-independent mappings 
u{'x{n) , g{n)) that produce the withdrawal vector y{n) described in line 9 
above. 

Lemma 6. LCSF/LCQ is not an MB policy. 

To prove lemma |6] we present the following counter example. Consider a 
system with L = 4 and K = 7. At time slot n the system has the following 
configuration: 

The queue state at time slot n is x(?t,) = (5,5,5,4). Servers 1 to 6 are 
connected to queues 1, 2 and 3 and server 7 is connected to queues 1 and 4 
only. 

Under this configuration, we can show that the LCSF/LCQ algorithm will 
result in ii{n) = (0, 2, 3, 3, 4) (where the first element represents the dummy 
queue that by assumption holds no real packets) and Kn{LCSF / LCQ) = 18. 



22 



A policy vr can be constructed that selects the feasible server allocation q = 
(1, 2, 3, 1, 2, 3, 4) which yields the state x(r2) = (0, 3, 3, 3, 3) and K„(7r) = 12 < 
Kn{LCSF/LCQ). Therefore, LCSF/LCQ is not an MB policy. 

The LCSF/LCQ policy is of particular interest for the following reasons: 
(a) It follows a particular server allocation ordering (LCSF) to their longest 
connected queues (LCQ) and thus it can be implemented using simple se- 
quential server allocation with low computation complexity, (b) the selected 
server ordering (LCSF) and allocation (LCQ) intuitively attempt to reduce 
the size of the longest connected queue thus reducing the imbalance among 
queues, and, (c) as we will see in Section |8| the LCSF/LCQ performance is 
statistically indistinguishable from that of an MB policy (implying that the 
counterexamples similar to the one in Lemma [6] proof have low probability 
of occurrence under LCSF/LCQ system operation). Therefore, LCSF/LCQ 
can be proposed as an approximate heuristic for the implementation of MB 
policies. 

7.2 Approximate Implementation of LB Policies 

In this section, we present the MCSF/SCQ policy as a low complexity ap- 
proximation of LB policies. We also provide an implementation algorithm 
for MCSF/SCQ using the same sequential server allocation principle that we 
used in Algorithm [T] above. 

The Most Connected Server First /Shortest Connected Queue (MCSF/SCQ) 
policy is the server allocation policy that allocates each one of the K servers 
to its shortest connected queue (not counting the packets already scheduled 
for service) starting with the most connected server first. The MCSF/SCQ 
implementation algorithm is analogous to Algorithm [T] except for lines 4 and 
5 which are described next: 
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Algorithm 2 (MCSF/SCQ Implementation). 
1. for t = 1,2,... do 



4. for j = K to 1 ; Servers in descending order 

5. Q[j] = min / : / G < argmin(X^|X^ > 0) 



10. ; End of Algorithmic 

Comments analogous to the ones valid for Algorithm [T] are also valid for 
Algorithm [2j 



8 Performance Evaluation and Simulation Re- 
sults 

We used simulation to study the performance of the system under the MB/LB 
policies and to compare against the system performance under several other 
policies. The metric we used in this study is EQ = E{J2i=i ^i): the average 
of the total number of packets in the system. 

We focused on two groups of simulations. In the first, we evaluate the 
system performance with respect to number of queues (L) and servers [K) 
as well as channel connectivity (Figures [s] to 7|). A rrivals are assumed to be 
i.i.d. Bernoulli. In the second group (Figures 8(a) to 8(c) ) we consider batch 
arrivals with random (uniformly distributed) burst size. 

The policies used in this simulation are: LCSF/LCQ, as an approximation 
of an MB policy; MCSF/SCQ, as an approximation of an LB policy. An MB 
policy was implemented using full search, for the cases specified in this sec- 
tion, and its performance was indistinguishable from that of the LCSF/LCQ. 
Therefore, in the simulation graphs the MB and LCSF/LCQ are represented 
by the same curves. This statement is also true for LB and MCSF/SCQ poli- 
cies performances. Other policies that were simulated include the random- 
ized. Most Connected Server First/Longest Connected Queue (MCSF/LCQ), 
and Least Connected Server First /Shortest Connected Queue (LCSF/SCQ) 
policies. The randomized policy is the one that, at each time slot, allocates 
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Figure 3: Average total queue occupancy, EQ, versus load under different 
policies, L = 16, = 16 and p = 0.2. 

each server randomly and with equal probability to one of its connected 
queues. The MCSF/LCQ policy differs from the LCSF/LCQ policies in the 
order that it allocates the servers. It uses the exact reverse order, starting the 
allocation with the most connected server and ending it with the least con- 
nected one. However, it resembles the LCSF/LCQ policies in that it allocates 
each server to its longest connected queue. The LCSF/SCQ policy allocates 
each server, starting from the one with the least number of connected queues, 
to its shortest connected queue. The difference from an LCSF/LCQ policy 
is obviously the allocation to the shortest connected queue. This policy will 
result in greatly unbalanced queues and hence a performance that is closer 
to the LB policies. 

Figure |3] shows the average total queue occupancy versus arrival rate un- 
der the five different policies. The system in this simulation is a symmetrical 
system with 16 parallel queues (L = 16), 16 identical servers {K = 16) 
and i.i.d. Bernoulli queue-to-server (channel) connectivity with parameter 



ends with a rapid increase. This abrupt increase happens at a point where 
the system becomes unstable. In this case, the queue lengths in the system 
will grow fast and the system becomes unstable. The graph shows that 
LCSF/LCQ, the MB policy approximation outperform^ all other policies. 
It minimizes EQ and hence the queuing delay. We also noticed that it 

^99% confidence intervals are very narrow and would affect the readability of the graphs. 
Therefore they are not included. 




25 



maximizes the system stability region and hence the system throughput as 
well. The MCSF/SCQ performed the worst. As expected, the performance 
of the other three policies lies within the performance of the MB and LB 
policies. 

The MCSF/LCQ and LCSF/SCQ pohcies are variations of the MB and 
LB policies respectively. The performance of MCSF/LCQ policy is close to 
that of the MB policy. The difference in performance is due to the order of 
server allocation. On the other hand, the LCSF/SCQ policy shows a large 
performance improvement on that of the LB policy. This improvement is a 
result of the reordering of server allocations. 

Figure [3] also shows that the randomized policy performs reasonably well. 
Moreover, its performance improves as the number of servers in the system 
decreases, as the next set of experiments shows. 

8.1 The Effect of The Number of Servers 

In this section, we study the effect of the number of servers on policy perfor- 
mance. Figure |4] (/T = 8) and Figure |5] (/T = 4) show EQ versus arrival rate 
per queue under the five policies, in a symmetrical system with L = 16 and 
p = 0.2. Comparing these two graphs to the one in Figure [3} we notice the 
following: 

First, the performance advantage of the LCSF/LCQ (and hence of an 
MB policy) over the other policies increases as the number of servers in 
the system increases. The presence of more servers implies that the server 
allocation action space is larger. Selecting the optimal (i.e., MB) allocation, 
over any arbitrary policy, out of a large number of options will result in 
a better performance as compared to the case when the number of server 
allocation options is less. 

Second, the stability region of the system becomes narrower when less 
servers are used. This is true because fewer resources (servers) are available 
to be allocated by the working policy in this case. 

Finally, we notice that the MCSF/LCQ performs very close to the LCSF/LCQ 
policy in the case of A' = 4. Apparently, when K is small, the order of server 
allocation does not have a big impact on the policy performance. 
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Figure 4: Average total queue occupancy, EQ, versus load, L = 1Q,K = 9, 
and p = 0.2. 
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Figure 5: Average total queue occupancy, EQ, versus load, L = 16,K = A 
and p = 0.2. 

8.2 The Effect of Channel Connectivity 

In this section we investigate the effect of channel connectivity on the perfor- 
mance of the previously considered policies. Figures [6] and [7] show this effect 
for two choices of L and K. We observe the following: 

First, we notice that for larger channel connection probabilities {p > 0.9), 
the effect of the policy behavior on the system performance becomes less 
significant. Therefore, the performance difference among the various policies 
is getting smaller. The LCSF/LCQ policy still has a small advantage over 
the rest of the policies, even though it is statistically difficult to distinguish. 
MCSF/SCQ continues to have the worst performance. As p increases, the 
probability that a server will end up connected to a group of empty queues 
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will be very small regardless of the policy in effect. In fact, when the servers 
have full connectivity to all queues (i.e., p = 1.0) we expect that any work 
conserving policy will minimize the total number of packets in a symmetrical 
homogeneous system of queues since, any (work-conserving) policy will be 
optimal in a system with full connectivity. 

Second, from all graphs we observe that there is a maximum input load 
that results in a stable system operation (maximum stable throughput)]^ An 
upper bound (for stable system operation) for the arrival rate per queue a 
is given by 

«<f (l-(l-#), (29) 

i.e., the average number of packets entering the system (aL) must be less 
than the rate they are being served. When p = 1.0, the stability condition 
in Inequality (29) will be reduced to aL < K, which makes intuitive sense 
in such a system. 

Finally, we observe that the MCSF/LCQ policy performance is very close 
to that of LCSF/LCQ. However, its performance deteriorates in systems with 
higher number of servers and lower probabilities for queue-server connectivity. 
It is intuitive that with more servers available, the effect of the order of 
server allocations on the policy performance will increase. Since MCSF/LCQ 
differs from LCSF/LCQ only by the order of server allocation, therefore, more 
servers implies larger performance difference. Also, the lower the connectivity 
probability, the higher the probability that a server will end up with no 
connectivity to any non-empty queue, and hence be forced to idle. 



8.3 Batch Arrivals With Random Batch Sizes 

We studied the performance of the presented policies in the case of batch ar- 
rivals with uniformly distributed batch size, in the range {1, . . . ,U}. Figure 
|8] shows EQ versus load for three cases with U = 2,5, 10, and hence average 
batch sizes 1.5, 3, and 5.5. The LCSF/LCQ policy clearly dominates all 
the other policies. However, the performance of the other policies, including 
MCSF/SCQ (LB approximation) approaches that of the LCSF/LCQ pol- 

^The last point in every curve corresponds to an overloaded system operating beyond 
its stability region. As a result the simulation is permanently in a "transient" state. Such 
points are shown in the presented graphs for illustrative purposes in order to show the 
trend of the queue size. 
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Figure 6: Average total queue occupancy, EQ, versus load under different 
policies, L = 8 and K = 4. 

icy as the average batch size increases. The performance of all the policies 
deteriorates when the arrivals become burstier, i.e., the batch size increases. 

9 Final Remarks 

The model and the results presented in this article can be regarded as a 
generalization (with the obvious added complexity as well as utility of our 
model) of the models and results reported by [3], [6], and [7]. 

In |3j , the authors investigated the optimal scheduling policy for a model 
of L parallel queues and one randomly connected server. This model is a 
special case of the model we presented in this article, i.e., when K = 1. Using 
stochastic dominance techniques, they proved that LCQ is optimal in that it 
minimizes the total number of packets in the system. In our work, we also 
use stochastic dominance techniques to prove the optimality of MB policies 
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Figure 7: Average total queue occupancy, EQ, versus load under different 
policies, L = 12 and K = A. 

for a wide range of cost functions (cost functions that are monotone, non- 
decreasing with respect to the partial order -<p) including the total number 
of packets in the system. It can be easily shown that for the case of a single 
server (i.e., K = 1) the LCQ policy minimizes the imbalance index and 
therefore, LCQ belongs to the set of MB policies. 

In [6] , the authors investigated the optimal policy for a model of L parallel 
queues with a stack of K servers. Each queue is randomly connected to 
the entire server stack. Only one server can be allocated to a queue at 
any time slot. In contrast, our model assumes independent queue-server 
connectivity, i.e., a queue can be connected to a subset of the K servers and 
not connected to the rest at any given time slot. We also allow for multiple 
servers to be allocated (when connected) to any queue. Therefore, the model 
in ^ can also be considered as a special case of our model, i.e., by letting 
Qi^jif) = 9iit),^h j,t and by adding the feasibility constraint yi(t) < 1. They 
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Figure 8: Average total queue occupancy, EQ, versus load, batch arrivals, 
L = 16 andK = 16. 

proved that a policy that allocates the K servers to the K longest connected 
queues (LCQ) is optimal. Under the constraints above, this policy would 
also minimize the imbalance index among all feasible policies, i.e., this policy 
belongs to the set of MB policies. 

In [7] the authors proved that, in a model of two parallel queues (L = 2) 
and multiple randomly connected servers, a MTLB (maximum through- 
put/load balancing) policy minimizes the expected total cost. They defined 
the cost as a class of functions of the queue lengths for the two queues in 
the system. In our work, we generalize the model in [7] as follows: (a) we 
extend the model to L > 2, (b) we optimize the cost function in the stochas- 
tic order sense which implies the expected total cost used in [7], and (c) we 
relax the supermodularity and convexity constraints that they enforced on 
the cost function, i.e., we prove our results for a larger set of cost functions 
that includes theirs. 

The authors of [7] defined the MTLB policy as the one that minimizes 
the lexicographic order of the queue length vector while maximizing the in- 
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stantaneous throughput. We can show that MTLB pohcy belongs to the 
set of MB pohcies. To do that, we have to show that a pohcy which mini- 
mizes the lexicographic order: (a) also minimizes the imbalance index, i.e., 
it belong to the set of MB policies, and (b) is a work-conserving policy. A 
work- conserving policy minimizes the number of idling servers and hence, 
it maximizes instantaneous throughput (by the definition of instantaneous 
throughput). Lemma [7] states these results formally. 

Lemma 7. Given the state (x(n),g(n)) during time slot n. Let A* be a 
vector resulted from the feasible withdrawal vector y*{n) G 3^(x(n), g(rz)). 
Suppose that X* <iex A for all feasible A. Then: a) the vector A* achieves the 
minimum imbalance index among all feasible vectors, and b) A policy that 
selects y*{n) is a work- conserving policy. 

Proof, a) Assume to the contrary that A* does not minimize the imbalance 
index. Then there must exist a y'(n) G 3^(x(n), g(n)), y'(n) 7^ y*('^) such 
that the imbalance index of the vector A' is strictly less than that of A*. 
This implies that a policy vr* that results in the withdrawal vector y*(n), 
and therefore the vector A*, belongs to the set 11^ \ n„ for some h > 0, i.e., 
it does not have the MB property during time slot n. For any given state, 
a policy that minimizes the imbalance index must exist, 'minimization on 
a finite set'. According to Lemma 4, h^^* balancing interchanges (Bis are 
feasible interchanges) are required to make any policy in U'^ belongs to 11^. 
Lemma D-1 shows that such balancing interchanges are feasible. Therefore, 
the following balancing interchange is both feasible and enhancing (it reduces 
the imbalance index): 

A' = A* -I(/,s), (30) 

for any /, s G {0, 1, . . . , L} and A* < A^* - 1. 

In other words, we perform a feasible server reallocation from the s*'^ 
longest queue to the l^'^ longest queue in the system during time slot n. The 
resulted leftover vector A' is related to the vector A* as follows: 

A; = Ar-i, a; = a: + i, a: = a*, yz^i,s. (31) 

Since Z < s by definition, then it is clear that A' <iex A*. This contradicts 
the initial assumption. Therefore, A* must have the minimum imbalance 
index. 

b) A feasible interchange y'{n) = y{n) + 1(/, 0) is a balancing one, since 
by definition of queue and the interchange feasibility conditions, we have 
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Xf{n) > Xo{n) + 1. Queue is permanently connected to all servers by 
assumption. According to Lemma B-1 this interchange will definitely reduce 
the imbalance index. Therefore, any policy that intentionally idles servers can 
always be improved (i.e., reduce its imbalance index) by using the balancing 
interchange 0) for some queue / e {1, . . . , L}. 

Prom part a) of this lemma, we showed that a policy that minimizes the 
lexicographic order will also minimize the imbalance index. We also showed 
that a policy that idles servers intentionally can not achieve the minimum 
imbalance index. Therefore, only a work-conserving policy can minimize the 
lexicographic order. □ 

From the above, we conclude that the MTLB belongs to the class of MB 
policies. 



10 Conclusion 

In this work, we presented a model for dynamic packet scheduling in a multi- 
server systems with random connectivity. This model can be used to study 
packet scheduling in emerging wireless systems. We modeled such systems 
via symmetric queues with random server connectivities and and Bernoulli 
arrivals. We introduced the class of Most Balancing pohcies. These policies 
distribute the service capacity between the connected queues in the system 
in an effort to "equalize" the queue occupancies. A theoretical proof of the 
optimality of MB policies using stochastic coupling arguments was presented. 
Optimality was defined as minimization, in stochastic ordering sense, of a 
range of cost functions of the queue lengths. The LCSF/LCQ policy was 
proposed as good, low-complexity approximation for MB policies. 

A simulation study was conducted to study the performance of five differ- 
ent policies. The results verified that the MB approximation outperformed 
all other policies (even when the arrivals became bursty). However, the 
performance of all policies deteriorate as the mean burst size increases. Fur- 
thermore, we observed (through simulation) that the performance gain of the 
optimal policy over the other policies is reduced greatly in this case. Finally, 
we observed that a randomized policy can perform very close to the optimal 
one in several cases. 
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Appendix .1 Proof of Lemma [l] 



Proof. To prove part (a), assume that D 
have: 



0; then, using Equation (10), we 



x(n) - y{n) 



x(n) - y 



From Equations (A-1) and rtT^, we have that 



MB, 



n) 



(A-1) 

fTT*^^) and thus vr 



has the MB property during time slot n. 

To prove part (b), assume that vr has the MB property at time slot n. 
Therefore, Hn{'^) = i^nij^^^^)- From Lemma Appendix .2.1 this is only pos- 
sible if either: (i) x(n) = x*^'^(n), or (ii) x(n) is obtained by performing a 
balancing interchange between the pair of the l^^ and the s^^ longest queues 



(/ < s) in yi [n] such that 



n] 



X[s\{n) + 1, is satisfied; note that 



there may be multiple such queue pairs. The balancing interchange in case 
(ii) will affect the length of two queues only (call them i and j) such that 
Xi{n) = xf^^{n) — 1 and Xj{n) = Xj^^{n) + 1, where i = [I] and j = [s] (for 
each given pair). Therefore, 

yi{n) = Xi{n) - Xi{n) = Xi{n) - {xf^{n) - 1) 



Xi[n] 
+ 1, 



(A-2) 



and. 



in) 



n) + 1) 



-- Xj{n)-Xj{n) 

= z/rw-1, 

while withdrawals from the remaining queues will be the same, i.e. 



yb[n) 



yf^(n),VM^,J- 



(A-3) 



(A-4) 



From Equations (A-2) through (A-4), we conclude that the vector D has 
components that are 0, -|-1, or —1 only. □ 



Appendix .2 Balancing Interchanges and the Imbal- 
ance Index 

In this section, we present a lemma that quantifies the effect of performing 
a balancing interchange on the imbalance index Kniir). 
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Lemma Appendix .2.1. Let x and x* be two L + 1- dimensional ordered 
vectors (in descending order); suppose that x* is obtained from x by per- 
forming a balancing interchange s) between two components, I and s, of 
X, where xi > Xg, such that, s > l;xi > Xa.'^a > I and Xg < Xh,Wb < s. Then 

L L+1 L L+1 

^^i' ~ 4) = J2J2^^^~ 

i'=l j'=i'+l i=l j=i+l 

-2(s - /) ■ l{.,>.,+2} (A-5) 

Proof. We generate the vector x* by performing a balancing interchange of 
two components, / and s (i.e., the l^^ and the s*^ largest components), in the 
vector X and reorder the resulted vector in descending manner. The resulted 
vector X* is characterized by the following: 

xl = Xk, I, s,l' , s' (A-6) 

where /' (respectively s') is the new index (i.e., the order in the new vector 
X*) of component / (respectively s) in the original vector x. 

From Equation (A-6) we can identify L — 2 elements that have the same 
magnitude in the two vectors x and x*. Therefore, the sum of differences 
between these L — 2 elements in both vectors will also be the same, i.e., 

L L+1 L L+1 

E E (^*' - 4) = E E (^^ - ^^•) 

i'=l j'=i'+l j=i, i=«+i 

i'(^{i',s'} j'(^{i',s'} mi,^} j^{i,s} 

We calculate the sums for the remaining terms (i.e., when at least one of 
the indices i,j belongs to {I, s} and/or i',j' belongs to {I', s'}) next. We first 
assume that xi > Xg + 2; in this case, we can easily show that /' < s'. Then, 
we have the following five, mutually exclusive, cases to consider: 

1. When i' = l',i = l,j' = s' and j = s. This case occurs only once, i.e., 
when decomposing the double sum in Equation (A-5) we can find only 
one term that satisfies this case. From Equation (A-6) we have 

X^f ~~ ^ ^ ('^~^) 
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2. When i' = = I, j' ^ s' and j 7^ s. There are L — I terms that satisfy 
this case. Analogous to case 1) we determined that 



Xi — Xj — 1 



(A-9) 



3. When i' ^ I, j' = s' and j = s. There are s — 2 terms that satisfy 

this case. In this case we can show that 



Xi Xa 



(A-10) 



4. When i' 7^ l',s',i 7^ l,s,j' = I' and j = I. There are / — 1 terms that 
satisfy this case. In this case we can show that 



Xi - Xi + 1 



(A-11) 



5. When i' = s',i = s,j' 7^ s' and j 7^ I, s. There are L — s + 1 terms 
that satisfy this case. In this case we have 



* * I 1 

/yt ryt rp /y , _]_ I 



(A-12) 



The above cases (i.e., Equations (A-7)-(A-12)) cover all the terms in 
Equation (A-5) when xi> Xs + 2. Combining all these terms yields: 

L L+l L L+l 

~ 4) = Y^^^i- ^3) 

i'=l j'=i'+l i=l j=i+l 

-2 ■ (l)-l-(L-/)-l-(s-2) + l-(Z-l) + l-(L-s + l) 

L L+l 
i=l j=i+l 



(A-13) 



Furthermore, if xi = Xg + 1, then from Equation (A-6) it is clear that 
X^/ —— Xg and X*, = xi, i.e., the resulted vector is a permutation of the original 
one. Therefore, the sum of differences will be the same in both vectors and 
Equation (A-5) will be reduced to 

L L+l L L+l 

E E (^*' - 4) = E E - ^^■) (A-14) 

i'=l j'=i'+l i=l j=i+l 

Equation (A-5) follows from Equations (A-13) and (A-14). □ 
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Appendix .3 Proof of Lemma [3] 

We first introduce a few intermediate lemmas tliat describe properties of 
and D. 

Lemma Appendix .3.1. The feasible interchange 0),/ > is a bal- 
ancing interchange. 

Proof. By definition, Xo{n) = 0. Since yo{n) > 0, therefore Xo{^) = Xo{n) — 



Uoin) < 0. According to tlie feasibility constraint (20), the interchange 0) 
is feasible only when Xf{n) > 1. Therefore, Xf{n) > xo{n) + 1, and it follows 
that 0) is a balancing interchange. □ 

Lemma Appendix .3.2. For a given policy n G n„_i and a time slot n, 

L L 

J2 A • i{A>o} = -Y.^r Md,<o} (A-15) 

i=0 j=0 

i.e., the sum of all positive elements o/ D equals the sum of all negative 
elements o/D. Moreover, Yli=o is an integer between and K. 

Proof. For any feasible withdrawal vector y(n), we have from Equation Q 
that 

L 

^Viin) = K, 

i=0 



where K is the number of servers. From equation (10), we have then: 

i=0 i=0 1=0 

= K-K = 0, 



and Equation (A-15) follows. The last assertion of the lemma follows from 



L L L 

5Z I A| = ^Di- 1{D,>0} - ^Dj- 1{D,<0} 
1=0 i=0 3=0 

L 

= 2 ■ A • iiD,>oi 

2=0 
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and 



L L 



i=0 i=0 
L 



i=0 i=0 

□ 

Lemma Appendix .3.3. Consider a given state (x(n),g(n)) during time 
slot n. Let f,t & {0, 1, . . . , L} be any two queues such that I(/, t) is feasible. 
A policy vr G n that results in Xt{n) < Xf{n) — 2 does not have the MB 
property at time n. 

Proof. The interchange t) is a balancing interchange by definition. Since 
Xt{n) < Xf{n) — 2, then the balancing interchange l{f,t) reduces the imbal- 



ance index by a factor of two according to Equation (23). Therefore, vr does 



not achieve the minimum imbalance index during time slot n, q.e.d. □ 

Lemma Appendix .3.4. Given the state (x(?2),g(n)) and a feasible with- 
drawal vector y{n) then a withdrawal vector y'{n) that results from perform- 
ing any sequence of feasible, single-server reallocations on y(n) is feasible. 



The proof of Lemma Appendix .3.4 is straightforward and therefore it is 
not included here. 

Lemma Appendix .3.5. Consider the state (x(n),g(n)) and any two feasi- 
ble withdrawal vectors y{n) andy'{n). Then, starting from y{n), the vector 
y'{n) can be obtained by performing a sequence of feasible, single-server re- 
allocations. 

Proof. To prove this lemma we construct one such sequence next. 

Let q(r;,),q'(n) denote two server allocations for the implementation of 
y{n),y'{n) respectively. Then we can relate y{n) and y'{n) as follows: 

K 

y'{n)=y{n) + J2M{n),qk{n)) (A-16) 

k=l 

since we assume both y{n) and y'{n) to be feasible, server k must be con- 
nected to both queues qk{n) and q'^in). Therefore, each interchange I(g^(n), qk{r 
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is equivalent to a feasible, single-server reallocation. Note that qk{n) = q[{n) 
is possible, for some k, in which case I(g^(n), qk{n)) = 0. By construction, all 
the interchanges in the right hand side of Equation (A-16) are feasible. □ 



We are now ready to prove Lemma [3] of Section 4.5 



Proof (Lemma^. We consider the following three cases assuming f ^ t: 

Case 1: / = 0. This case is not possible by contradiction. By assumption, 
Dq > +1, which means that y^^{n) > yQ{n) + l. This case states that an MB 
policy idled at least one more server than vr. Therefore, x^^^in) < -1. This 
makes queue the shortest queue. Allocating the idled server to queue t, i.e., 
the interchange 0), is both feasible (since y(n) is feasible by assumption) 
and balancing (by Lemma Appendix .3.1[ ). The interchange I(t, 0) will result 
in a withdrawal vector y'(n) = y^^{n) + I(t, 0). Let s be the order of queue 
/ = when ordering the vector x*'^'^(n) in a descending manner. Therefore, 
s = L-|- 1. Furthermore, in order for I(t, 0) to be feasible queue t must not be 
empty (according to feasibility constraint (20)) which implies that xf^^{n) > 



1 and the order of queue t is I < s. Therefore, xj (n) < [n) — 2 and 
the interchange I(t, 0) will reduce the imbalance index by 2(s — /) according 



to Equation (23). This implies that the new policy has a smaller imbalance 
index than an MB policy. This contradicts the fact that any MB policy 
minimizes the imbalance index. 

Case 2: t = 0. When t = then the interchange I(/, t) is the process 
of allocating an idled server to queue / > 0. This, according to Lemma 
Appendix .3.1[ is a balancing interchange. 

Case 3: t, / > 0. We will show that this case will also result in a balancing 
interchange. Let y{n) be the original withdrawal vector. Let y*{n) be the 
withdrawal vector resulted from the feasible interchange I{f,t), i.e.. 



[n 



y(^) + i(/,t) 



Using the assumption Dt < —1 and Equation (16), we arrive at the 
following: 

y^^^{n)-y,in) < -1 

yf^(n) < y:{n)=y,{n)-l (A-17) 



hence, 



- yt '^{n)>xt{n) - y*{n) = Xt{n) - {yt{n) - 1) 
xf^{n)>xl{n) = xtin) + 1, t > (A-18) 
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Similarly, using the assumption Df > +1 and Equation (15), we have 

yf^H-y^n) > +1 

yrin) > y}in) = yfin) + l (A-19) 

hence, 

.MB I 



^f{^) -Vf in)<Xf{n)-y*j{n) = Xf{n) - {yf{n) + 1) 



X 



MB, 
f ' 



n) <x*jr{n) = Xf{n) 



f>0 



(A-20) 



To show that t) in this case is a balancing interchange, we have to 
show that Xf{n) > Xtiji) + 1. Suppose to the contrary that Xf{n) < Xt{n); 
then, from Equations (A-18) and (A-20), we have 



Xf{n) < Xt{n) 
x*f{n) + 1 < Xt{n) - 1 
x}{n) < x;{n)-2 



(A-21) 



From (A-18) and (A-20) we have 



xf^{n) < x}{n) < x;{n) 



2<xr{n) 



X 



MB 



in) < xrin) 



(A-22) 



The differences satisfy the inequalities -D/ > +1 and Dt < —1 by as- 
sumption, i.e., there is at least one more (respectively one less) server al- 
located to queue / (respectively queue t) under the MB policy. Therefore, 



y*(n) = y^^{n) + I{t, /) is feasible and Inequality ( A-22 ) is a contradiction 



according to Lemma Appendix .3.3, We must have that Xf{n) > Xt{n) + 1 
and by definition the interchange l{f,t) is a balancing interchange. □ 



Appendix .4 Proof of Lemma [4] 

We present the proof of Lemma|4]of Section [475] in this appendix. Lemma Ap- 



pendix .4.1 , stated below, guarantees the existence of a feasible interchange. 



Lemma Appendix .4.1. Consider a given state (x(n),g(n)) and a policy 
TT G n„_i that selects a withdrawal vector y{n) during time slot n. Let F,T 
denote the (non-empty) sets of queues for which Df > +1 and Dt < —1, 
respectively. Then, there exist at least two queues f E F and t E T such that 
the interchange I(/, t) is feasible. 
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Proof. Let vr* G Yl'^^ be an MB policy that selects the withdrawal vector 
y*{n) during time slot n. Let D = y*(n) — y(n). Furthermore, let q*(n) and 
q(n) be two implementations of y*{n) and y{n) respectively. From Lemma 
Appendix .3.51 we have: 



K 



[n 



(A-23) 



k=l 



The summation on the right-hand side of Equation (A-23) is composed 



of K terms, each of which represents a reallocation of a server k from queue 
gfc(n) to queue ql{n). Such server reallocation can be formulated as an inter- 
change I{ql{n),qkin)). 

In the following, we will selectively use i out of the K terms of the sum- 
mation in Equation (A-23) to construct a feasible interchange I(ri,rj+i) = 



I('^i5 ^^2) +I('"2, '"3) + - ■ ■ + I('"i, fi+i) for some i < K, with ri E F and rj+i G T. 
We will show that such a queue rj+i, i < K that belongs to T does exist and 
the interchange I(ri,rj_|_i) is feasible. 

Let ri G F, ri G {1, 2, . . . , L} then using Equation (A-23) we can write 



K 



fc=l 



Since ri G -F by our assumption, then we have D^.^ > 1 and 
From Equations (A-24) and ( |A-25 ) we conclude 



(A-24) 



(A-25) 



K 



K 



(A-26) 



In words, there is at least one more server allocated to queue ri under vr* 
than the servers allocated to queue ri under vr. Let ki be one such server. 



From (A-26) we conclude that one of the K terms in Equation (A-23) must 



be I{ql^{n),qk^{n)) such that ql^{n) = ri,qk^{n) = r2,/ci G {1,2,...K}. In 
other words, a server ki and two queues ri = ql^iji) and r2 = qk^in) must 
exist such that the interchange I(ri,r2) is feasible. 



41 



The feasibility of I(ri,r2) stems from the fact that server ki is allocated 
to queues ri and r2 under two different policies, namely ir* and vr. This is 
possible only if 

fl'ri,fci(^) = 9r2Mi^) = 1 (A-27) 



Furthermore, using Equation (A-25) we can write 

> (n) + 1 



n) 



or Xr^{n) - y*^{n) < Xr^{n) - VrM) - 1 
hence, < x*^{n) < — 1 



From Equation (A-28) we conclude 

(n) > 1 



(A-28) 



(A-29) 



Equations (A-27) and (A-29) are sufficient for the feasibility of the inter- 
change y{n) + I(ri, r2). 

Consider queue r2 above. One of the following two cases may apply: 

Case (1) r2 G T: The proof of the lemma in this case is completed by 
letting / = ri and t = r2. The resulted interchange I(/, t),/ E F,t E T is 
feasible by construction and the lemma follows. 

Case (2) r2 ^ T: Define y^(n) as follows: 



y^(ra) = y{n) + I(ri,r2) 



(A-30) 



Using Lemma Appendix .3.4 



drawal vector. From Equations (A-30) and (13) we can write 



we conclude that y^(^) is a feasible with- 

(A-31) 



From Equation (A-31) and since r2 ^ T we conclude 

y:,{n)>yr,{n)>yl^{n) + l (A-32) 

Therefore, one of the terms of the summation in Equation ( |A-23[ ) must be 
a server reallocation of some server k2 from queue = q^^ (n) to queue r2 = 
qX^{n), i.e., the interchange I(r2,r3) is a feasible, single-server reallocation. 
It follows that y^(n) is feasible, where 



y'(^) 



y^{n) +I(r2,r3) 

y(n) + I(ri,r2) + \{r2,n) 

y{n) + I(ri,r3) 



(A-33) 
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If r3 G T then we complete the proof using the argument in case (1) 
above. Otherwise, we repeat the argument in case (2) again. 

Repeating the previous argument i times, 1 < i < i^, we arrive at the 
following relationship: 



y\n) = y{n) + ^ I(rj-, r^+i). 



(A-34) 



where by construction, each one of the i terms in Equation (A-34) above 



corresponds uniquely to one of the terms of the summation in Equation (A- 



23). For every i we check to see whether rj+i G T (in which case the lemma 



is proved) or not. If not then we have 

l/.V, (n) > vr^^, (n) > yi^^^ {n) + 1 (A-35) 

Repeating the argument K times (one for each term of the summation 
in Equation (A-23)) we will show that a queue rj+i G T, where I(rj,rj+i) is 
one of the terms in Equation (A-23), must exist. 

In order to do that, we assume to the contrary that rj+i ^ T, Vi = 
1, 2, ... i^. The K*^ (last) server reallocation I(ri^, rK+ij^rx = Q'fc^('^)5 ''^i^'+i = 
Qkni''^) will result in the withdrawal vector y^(n), such that. 



(A-36) 

Since there is one-to-one correspondence between the summation terms in 
Equation (A-36) and those in Equation (A-23) by construction, then we can 
write 



K 



y{n) + ^l{q*k{n),qk{n)) 



k=l 



hence y^{n) = y*{n). However, since tk+i ^ T then 



(A-37) 



(A-38) 



have a contradiction. We conclude that there must exist a queue rj+i G T 
such that server ki reallocation I(rj, rj+i), rj = ql.{n), rj+i = qkX^) is feasible. 
Let f = Ti and t = rj+i. It follows that the interchange I(/, t) = I(ri,rj_,_i) 
is feasible and the lemma follows. □ 
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Proof for Lemma^ From its definition and Lemma Appendix .3.2, /i^ = 
X^i^o 1-^*1/2 integer between and K. If = 0, tlien vr lias tlie MB 
property during time slot n according to Lemma [T] 

So, suppose that h^^ > 0. From Equation |A-15[ we can pair queues fi and 
ti,^ ^ i hj^, such that for every i, Dj. > +1 and Df. < — 1. From Lemmas 
Appendix .4.1 andlsl the interchange I(/j,tj) is feasible and balancing. 



Since we have h^, such pairs of queues, then applying the hj, balancing 
interchanges tj) described by Lemma [s] to policy vr will result in a policy 
71* for which D* = 0, i.e., y*{n) = y* (n). Hence the resulting policy 

□ 



Appendix .5 Coupling Method and the Proof of Lemma 

m 

Appendix .5.1 The Coupling Method 

If we want to compare probability measures on a measurable space, it is 
often possible to construct random elements [T] , with these measures as their 
distributions, on a common probability space, such that the comparison can 
be carried out in terms of these random elements rather than the probability 
measures. The term stochastic coupling (or coupling) is often used to refer to 
any such construction. In the notation of [1] , a formal definition of coupling 
of two probability measures on the measurable space {E, £) (the state space, 
e.g., E = n,n'^,Z+,etc.) is given below; further details regarding coupling 
method and its application can be found in fTj. 

A random element in [E, S) is a quadruple {Q, JF, P, X), where (f2, P) 
is the sample space and X is the class of measurable mappings from Q to E 
{X is an ii^-valued random variable, s.t. X~^{B) G ^ for all B e S). 

Definition: A coupling of the two random elements {Q, JF, P, X) and {Q', P', 
in {E,S) is a random element {Q,^,P, (X,X')) in (E'^,S'^) such that 

Xix and X' = X', (A-39) 
where = denotes 'equal in distribution'. 

Remark 4. The above definition makes no assumption about the distribution 
of the collection of random variables X; for example, X may be a sequence 
of non-i.i.d. random variables. □ 
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In the area of optimal control of queues, coupling arguments have been 
used extensively to prove characteristics of the optimal policies for many 
queuing systems, c.f. [19], [20], [3], [6^ and many others. 

We apply the coupling method to our proof as follows: Let u and vr be 
a given sample path of the system state process and scheduling policy. The 
values of the sequences {X{n)} and {Y{n)} can be completely determined 
by Lo and vr. We denote the ensemble of all random variables as system S. 
A new sample path, Cj and a new policy tt are constructed as we specify 
in the proof. We denote the ensemble of all random variables (in the new 
construction) as system S. Then, in the coupling definition, Cj = {co, u) and 



the "coupled" processes of interest in Equation (A-39) will be the queue sizes 
X = {X{n)} and X' = {X{n)}. 

We define u as the sequence of sample values of the random variables 
(X(1),G(1),Z(1),G(2),Z(2),...), i.e., co = (x(l), g(l), z(l), g(2), z(2), . . .). 
The sample path co = (x(l), g(l), z(l), g(2), z(2), . . .) is constructed such 
that (a) x(l) = x(l), (b) g{n) is the same as g(n), except for two elements 
that are exchanged, (c) z(n) is the same as z(n), except for two elements that 
are exchanged. The selection of the appropriate elements that are exchanged 
is detailed in the proof that follow^ 

The new policy vf is constructed (by showing how vr chooses the with- 
drawal vector y(-)) as detailed in the proof. Then using Equation ([T]), the 
new states x(-), x(-) are determined under vr and vf . The goal is to prove that 
the relation 

k{t) x(t) (A-40) 
is satisfied at all times t. Towards this end, the preferred order (introduced 



in Section 5.1) can be described by the following property: 



Property 1: x is preferred over x (x -<p x) if and only if one of the state- 



ments Rl, R2 or R3 (that we introduced in Section 5.1) holds. We restate 
these statements here for the sake of presentation: 

(Rl) X < x: the two vectors are component-wise ordered; 



(R2) X is a two-component permutation of x as described in Section 5.1 



'^In the system we are studying, {G{n),Z{n)} has the same distribution as 
{G(n),Z(n)}, since the distributions of G(n) as well as Z{n) are i.i.d. and will not 
change when their elements are reordered. The mappings from G(n) to G(n) and from 
Z(n) to Z{n) are one-to-one. 
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(R3) X is obtained from x by performing a "balancing interchange" as de- 
scribed in Section IsTTl 

Appendix .5.2 Proof of Lemma [5] of Section [5] 



Proof. Fix an arbitrary policy vr G 11^ and a sample path u = (x(l), g(l), z(l), 
where x(.), g(.) and z(.) are sample values of the random variables X(.), G(.) 
and Z(.). Let , tt* G 11^^^ be an MB policy that works on the same system. 
The policy tc* chooses a withdrawal vector y*{t),\/t. 

The proof has two parts; Part 1 provides constructions for u and vr (as 
defined by Lemma |5] statement) for times up to t = r. Part 2 does the same 
for t > T. 

Part 1: For the construction of a), we let the arrivals and channel states 
be the same in both systems at all time slots before r, i.e., z(t) = z(t) and 
g(t) = g(t) for all t < T. We construct vr such that it chooses the same 
withdrawal vector as vr, i.e., we set y(t) = y{t) for all t < t. In this case, 
at t = r, the resulting queue sizes are equal, i.e., x(r) = x(r). In the 
remainder of Part 1, we will construct the policy vf at time slot r such that: 
(a) TT G n^~^, i.e., vr is closer to tt* G n*^"^ than vr, and (b) the resulting 
queue length under vr is preferred over that under the original policy vr, i.e., 
x(r + 1) -<p x(r + 1). Condition (b) is necessary for proving the second part 
of the lemma, i.e., the domination of policy n over tt, a result that will be 
shown in Part 2 of this proof. 

At time slot r, let u have the same channel connectivities and arrivals as 
u, i.e., let g(r) = g(r) and z(r) = z(r). Furthermore, let D = ?/*(r) — |/(r). 
Recall that h = ^,^0 lA|/2. Then one of the following two cases may apply: 

1- During time slot t = t, the original policy tt differs from tt*, the MB 
policy, by strictly less than h balancing interchanges. Then tt G 11^^^ as well, 
so we set y(r) = y(r). In this case, the resulting queue sizes x(r + l), x(r + l) 



will be equal, property (Rl) holds true and (A-40) is satisfied at t = r + 1. 

2- During time slot t = t, n differs from the MB policy tt* by exactly h 
balancing interchanges. Since vr G 11^ and h > and following Lemma |4| we 
can identify two queues I and s such that: (a) Di > 1, (h) Dg < —1, and (c) 

s) is feasible. 
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The construction of vr is completed in this case by performing the inter- 
change s), i.e., 

y(r) = y(r)+I(/,s), (A-41) 

or equivalently, 

X(r) = x(r) - s) (A-42) 

According to Lemma |3| this interchange is balancing. To complete the 
construction of u, we examine the arrivals under u during time slot r. We 
set 5j(r) = Zi{T),'ii ^ l,s. For queues / and s, we do the following: (i) if 
xi{t) = Xs{t) + 1 and Zs{t) > zi{t) then we swap the arrivals for queues I and 
s, i.e., we let zi{t) = Zs{r) and Zs{t) = zi{t), (ii) otherwise, let zi{t) = zi{t) 
and Zs{t) = Zs{t). The queue lengths at the beginning of time slot (r -|- 1) 
under the two policies satisfy property (Rl) in case (i) and (R3) otherwise. 
In either case, (A-40) is satisfied at t = r + 1. 

Starting from a preferred state at t = r + 1, we will show next, in Part 
2 of the proof, that a feasible control at time slot t > t, such that the 
constructed policy tt dominates the original one vr, will always exist. We do 
that by showing one such construction. 

Part 2: In this part of the proof, we construct cD, vf for times t > t, such 
that the preferred order x(t) -<p x(t) is valid for all t > t. This will insure fr 
domination over vr. We will use induction to complete our proof. We assume 
that vf and u are defined up to time n — 1 and that x(n) -<p x(n). We will 
prove that at time slot n, vr can be constructed so that x(n + 1) -<p x(n + 1). 
Thus, we have to show that either Rl, R2 or R3 holds at time slot n + 1. 

The following three cases, corresponding to properties (Rl), (R2) and 
(R3) are considered next. 

Case (1) x(?T,) < x(n). The construction of u is straightforward in this 
case. We set z(n) = z{n) and g{n) = g{n). We construct vr such that 
y(n) = y(n). In this case, its obvious that x(n + 1) < x(n + 1) and (A-40) 
holds at t = n + 1. 

Case (2) St{n) is a permutation of x(n), such that St{n) can be obtained 
from x(?2) by permuting components i and j (as described in property R2 
of the preferred order). For the construction of u, we set gi,c{,n) = gj^dn) 
and gj,c{n) = gi^dn), for all c = 1, 2, . . . , i^; Zi{n) = Zj{n) and Zj{n) = Zi{n) 
(refer to footnote 7 or [6]); the connectivities and arrivals for each one of the 
remaining queues are the same as in u. We construct tt such that yi{n) = 
yj{n), Vjin) = yi{n) and ym{n) = ym{n) for all m ^ As a result, x(n-|-l) 
and x(?2 -|- 1) satisfy property (R2) and (A-40) is satisfied at t = n -|- 1. 
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Case (3) x(n) is obtained from x(ri) by performing a balancing inter- 
change for queues i and j as defined in property (R3). In this case Xi{n) > 
Xj{n) + 1, by the definition in (R3)|^ There are three cases to consider: 

(3. a) Xi{n) = Xj{n) + 1. Therefore, Xi{n) = Xj{n) and Xj{n) = Xi{n), i.e., 
the vectors x(n) and x(?t,) have components i and j permuted and all other 
components are the same. This case corresponds to case (2) above. 

(3.b) Xi{n) > Xj{n) + 1 and yiiji) < yjiji). We construct Cj as in case 
(1) above, and we let ym{n) = ym{n),\/m ^ j. Note that it is not feasible 
for policy vr to empty queue i in this case. Depending on whether vr empties 
queue j or not at t = n, the construction of vr will follow one of the following 
two cases: 

(i) yj{n) < Xj{n), i.e., tt does not empty queue j at t = n, then let yj{n) = 
yj(n) (i.e., TT is identical to vr at t = n). In this case, property (R3) will 



be preserved regardless of the arrivals patterrQ hence (A-40) is satisfied at 
t = n + l. 

(ii) yj{n) = Xj{n), i.e., vr empties queue j at t = n. Then if under policy 
vr all the servers connected to queue j are allocated, then let yj{n) = yj{n). 



As in case (i) above, property (R3) holds and (A-40) satisfied at t = n + 1. 

In the event that tt empties queue j without exhausting all the servers 
connected to queue j, then tt will be constructed such that one of these idling 
servers is allocated to queue j, i.e., yj{n) = yj{n) + 1, so that vf preserves the 
work conservation property at t = n. Since Xj{n) = Xj{n) + 1 by property 
(R3) and Zj{n) = Zj{n) by construction, then we have 

Xj{n + 1) = Xj{n + 1) = Zj{n) 

Since Xi{n) = Xi{n) — 1 by property (R3), Zi{n) = Zi{n) and yi{n) = yi{n) by 
construction, we have 

Xi{n + 1) = Xi{n + 1) - 1 
The rest of the queues will have the same lengths in both systems at t = 



n + 1. Therefore, (Rl) holds with strict inequality and (A-40) is satisfied at 
t = n+1. This case shows that a "more" balancing policy results in a strict 
enhancement of the original policy. 

Cases (i) and (ii) are the only possible ones, since vr cannot allocate more 
servers to queue j than its length. 



^By definition, we have Xi{n) > Xj{n), Xi{n) — Xi{n) — 1 and Xj{n) = Xj{n) + 1. 
^Note that if Xi = Xj + 1 then property (R2) is a special case of (R3). 
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(3.c) Xi{n) > Xj{n) + 1 and yiiji) > yj{n). We consider the following two 
cases: 

(i) Uiin) = Xi{n), i.e., tt empties queue i at t = n. To construct u for 
this case, we set z(n) = z{n), gm,c{^) ~ 9m,c{n) for all m 7^ and for all c. 
For queues i and j we do the following: 

Let server r be a server that is connected to queue i at time slot n such 
that qr{n) = i (i.e., server r is allocated to queue i by policy vr at t = n). 
Now, we switch the connectivity of server r to queue i and that of server r to 
queue j, i.e., we set Qj^riji) = Qi^riji) and gi^riji) = gj,r{n) (refer to footnote 
7 or pj). The rest of the servers will have the same connectivities to queues 
i and j under both pohcies, i.e., we set Qi^ciji) = gi,c{n) and Qj^cip) = djA^) 
for all c ^ r. 

We construct tt such that qr{n) = j and qdn) = qdn) ,\/ c ^ r . This means 
that vf differs from tt, at t = n, by one server allocation (server r) that is 
allocated to queue j (under vr) rather than queue i (under vr). From equation 
(|2]), we can easily calculate that the resulting queue lengths at t = n + 1 (for 
any arrivals pattern) will be: 

Xm{n + l) = Xrn{n + 1), W 771. 



It follows that property (Rl) is satisfied and therefore (A-40) is satisfied 
at t = n + 1. 

(ii) yi{7i) < Xj(n), i.e., tt does not empty queue i sX t = n. Then consider 
the following: 

If TT does not empty queue j at t = or if vr empties queue j and in 
the process it exhausts all servers connected to queue j, i.e., tt does not idle 
any server connected to queue j, then we construct uj and vr similar to case 
(3.c(i)) above and the same conclusion holds. 

If on the other hand, vr empties queue j without exhausting all its con- 
nected servers and therefore vr is forced to idle some of the servers connected 
to queue j, then let r' be one such server. We set z(n) = z(n), g(n) = g(n). 
We construct vr such that yjiji) = yj{n) + 1, by allocating server r' to queue 
i under tt, i.e., we set qr'iji) = j. This is feasible since Xj(n) = Xj(n) + 1 by 
property (R3). We also have Xi{n) = Xi{n) — 1 (by property (R3)). Since 
z(n) = z(?T,) by construction, then similar to case (3.b(ii)), property (Rl) 
holds with strict inequality at t = n + 1 for any arrivals pattern, and (A-40) 
follows. 

Since vr cannot allocate more servers to queue j than its length, therefore, 
(i) and (ii) are the only possible cases. 
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Note that policy vr belongs to 11^ ^ by construction in Part 1; its domi- 
nance over TT follows from relation (24). □ 
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